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The Honorable Robert Menendez 
United States Senate 
Washington, DC 20510 

Dear Senator Menendez: 

Thank you for your letter to Mr. Alex Azar, Secretary of the U.S. Department of Health and 
Human Services (HHS), regarding the Centers for Disease Control and Prevention’s (CDC) 
firearm injury data. Secretary Azar asked me to respond on his behalf. 

CDC appreciates your interest and concern and is committed to protecting the health, safety, and 
security of the American people. CDC strives to provide the most accurate information available, 
including data related to violence. In addition to collecting non-fatal injury data, CDC funds a 
state-based surveillance system, the National Violent Death Reporting System (NVDRS). 
NVDRS captures information on all violent deaths, including those with firearms listed as the 
mechanism of death. The fiscal year (FY) 2020 budget requested $24 million for NVDRS, and, 
thanks to an increase from Congress in FY 2018, CDC is expanding the system to all 50 states 
and Washington, DC. 

Enclosed with this response are answers to your questions related to CDC’s collection of non- 
fatal, firearm-related injury data. 

Thank you, again, for your letter and your interest in firearm injury data. If you have additional 
questions or concerns, please contact Cristi Schwarcz in the CDC W ashing ton Office at 
(202) 245-0600 or CSchwarcz@cdc.gov. This response is being sent to the co-signers of your 
letter. 


Sincerely, 



Robert R. RedfieldfMD 
Director, CDC 
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Centers for Disease Control and Prevention’s Responses to Questions about Firearm 

Injury Data 

1. What is the Centers for Disease Control and Prevention (CDC) doing to more 
accurately track non-fatal firearm injuries? Please he specific. 

CDC is exploring ways to improve the precision and accuracy of non-fatal firearm injury 
estimates available from the National Electronic Injury Surveillance System - All Injury 
Program (NEISS-AIP). This includes: 

• Investigating the cost of expanding the roster of hospitals participating in NEISS- 
AIP; 

• Examining the impact of recent changes to the sample due to hospitals dropping 
out and being replaced; 

• Analyzing data from the Healthcare Cost and Utilization Project Nationwide 
Emergency Department Sample (HCUP-NEDS) to get a 2016 firearms estimate 
(when available; current data only goes through 2014) for comparison to the 2016 
NEISS-AIP. Note that only preliminary definitions of the new ICD-10-CM 1 
coding scheme are available and they have not yet been integrated into HCUP- 
NEDS. This definition affects HCUP-NEDS data but not NEISS-AIP data; 

• Exploring the utility and cost of a separate Web-based Injury Statistics Query and 
Reporting System (WISQARS 2 ) firearm injury reporting module that includes all 
hospitals in NEISS. 

CDC has engaged the Consumer Product Safety Commission (CPSC) to explore the 
feasibility of implementing the first two improvements mentioned above. In addition, 
CDC is open to collaborating with CPSC on any improvements they may suggest to more 
accurately track non-fatal firearm injuries. 

2. What additional resources, if any, does CDC need in order to better ensure more 
accurate firearm injury data? 

With additional resources, CDC could potentially collaborate with CPSC to expand the 
roster of hospitals participating in NEISS-AIP. This would increase the sample size and 
likely improve the precision of the non-fatal firearm-injury estimates. It would also allow 
for comparison with other types of non-fatal injury data. 


3. Why does CDC use the NEISS database rather than the Healthcare Cost and 
Utilization Project database in order to track non-fatal firearm injuries? 

CDC uses the NEISS-AIP data rather than the HCUP-NEDS data to track non-fatal 
firearm injuries because: 


1 ICD-10-CM = International Classification of Diseases, Tenth Revision, Clinical Modification 

2 WISQARS is an interactive, online CDC database, available at www.cdc.gov/ininiv/wisqars that provides fatal and 
non-fatal injury, violent death, and cost of injury data. 
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« NEISS-AIP data include key narrative information not found in HCUP-NEDS. 

NEISS data coders, CPSC, and CDC review narrative information to determine the 
intent of the injury (i.e., whether it was unintentional or violence-related from an 
assault, legal intervention, or act of self-harm). In contrast, HCUP-NEDS data sets 
use ICD codes to indicate intent. HCUP-NEDS codes many potential assault injuries 
as “undetermined” intent and therefore does not count them as assault-related. This 
limits the utility of HCUP-NEDS data in assessing trends in assault-related firearm 
injuries. 

• NEISS-AIP data are available in a more timely fashion. In general, NEISS-AIP data 
are available through WISQARS approximately one year after entry in NEISS-AIP. 
For example, CDC posted 2017 NEISS-AIP data in January 2019. In contrast, as of 
April 2019, the most recent publically available HCUP-NEDS data on the HCUP 
online querying system is from 2014. 

• NEISS-AIP data captures information not found in HCUP-NEDS. For example, 
NEISS-AIP collects firearm injury data on the type of firearm, whether the injury was 
intentional or unintentional, whether or not a crime was involved, and whether or not 
drugs were involved. 

NEISS-AIP data produce stable estimates for most non-fatal injuries including 
unintentional injuries from firearms, falls, assaults, fires, and other sources. When users 
compare HCUP-NEDS and NEISS-AIP non-fatal firearm injury estimates from 2006 to 
2014, the estimates between the two systems do not differ statistically (i.e., the 
confidence intervals overlap). 


4. What accounts for the four-fold increase in CDC’s gun injury confidence interval 
between 2011 and 2017? 

CDC attributes recent increases in the violence-related firearm injury estimates and their 
relative uncertainty to two factors: shifts in the roster of participating hospitals and the 
weighting of cases in the data set. 

Sample data require the use of weights in order to account for the sampling design when 
producing estimates. In formulating national estimates, each case treated in an emergency 
department is assigned a weight based on the probability of selection, and includes 
factors such as hospital size (annual number of emergency department visits), hospital 
non-response, and the population included in the hospital capture area. Higher weights 
can contribute to increased estimates and greater relative uncertainty. Recently, several 
high-weight hospitals joined the roster of participating hospitals, replacing other similar 
hospitals no longer participating. This alone is not unusual. However, there was an 
accompanying shift in reported firearm-injury emergency department visits away from 
lower-weight hospitals and toward higher-weight hospitals. 3 This resulted in increased 
yearly estimates and greater relative uncertainty. 


3 For more information on weighting, please see The NEISS Sample (Design and Implementation), at 
www.cp sc. gov/s3fs-piibjic/pdfs/bik media 20Q1 dQ 1 l-6b6.pdf 
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A confidence interval will widen as an estimate increases, even when the relative 
uncertainty associated with the estimate remains constant. When the relative uncertainty 
also increases, as occurred for the most recent yearly estimates, the confidence interval 
will widen more. NEISS-AIP is still the best data we have on non-fatal firearm injuries. 
For CDC’s estimates, relative uncertainty is measured by the coefficient of variation 
(CV) as displayed in the CDC WISQARS reporting application. 


5. As the confidence interval increased over the course of 16 years, did CDC take any 
steps to ameliorate the problem? If so, please describe those steps. If not, please 
explain why no action was taken. 

The confidence interval gradually increased between 2001 and 2012, from a width of 
approximately 55,000 to a width of approximately 78,000. However, the relative 
uncertainty did not increase in a comparable manner, with CVs at 22.1 percent in 2001 
and 24.4 percent in 2012. There is nothing remarkable about the confidence intervals for 
these data from 2001 to 2012 (11 years). 

The confidence interval began to widen, accompanied by persistent increases in relative 
uncertainty, beginning in 2013, and most noticeably in 2016 and 2017. As mentioned 
previously, we attribute these differences to the hospitals included in the sample and the 
weighting shift that occurred over this time period. 

CDC is looking into ways to narrow confidence intervals and lower relative uncertainty 
for future estimates. As outlined in the answer to question 1, we are considering 
expanding the number of hospitals providing data for all types of injury, but this would 
require additional funding. By expanding the roster of participating hospitals, the 
influence of any one hospital should be reduced and more stable estimates should be 
attainable. 


6. What role has the Dickey Amendment played in CDC’s decision to continue using 
NEISS even as the data proved less and less reliable? 

The provision within Public Law 104-208 (often referred to as the “Dickey Amendment”) 
did not play a role in CDC’s scientific decision to use NEISS-AIP data. 

7. Has CDC received instructions from any executive branch officials pursuant to its 
non-fatal firearm injury data? If so, please describe any such instructions and 
communications. 

CDC has not received any instructions from any executive branch officials in relation to 
its collection, analysis, or publication of non-fatal firearm injury data. 
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